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ABSTRACT 

In  this  note  we  derive  norm  estimates  for  the  rate  of 
convergence  of  the  basic  alternating  direction  methods  for 
solving  elliptic  difference  equations.   The  equations  treated 
have  variable  coefficients  and  are  defined  in  arbitrary  regions. 
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A  NOTE  ON  THE  CONVERGENCE  OF 
ALTERNATING  DIRECTION  METHODS 

0 .    Introduction. 

Convergence  of  the  single  parameter  alternating  direction 
methods  of  Douglas,  Peaceman,  and  Rachford  [1],  [2],  [5]  has 
been  proved  for  a  wide  class  of  elliptic  difference  equations; 
see,  for  example,  Blrkhoff  and  Varga  [4].   The  proof  consists 
In  showing  that  a  certain  matrix,  similar  to  the  defining  matrix, 
has  spectral  radius  less  than  one.   Since  the  defining  matrices 
for  these  methods  are  symmetric  only  when  they  are  Induced  by  a 
proper  discretization  of  Laplace's  equation  In  a  rectangular 
region,  an  estimate  for  their  spectral  radii  does  not  Imply  a 
corresponding  norm  estimate  for  their  rate  of  convergence. 

The  purpose  of  this  note  Is  to  present  another  proof  of 
the  convergence  of  the  two  basic  alternating  direction  methods 
which,  at  the  same  time,  provides  a  norm  estimate  for  their 
rate  of  convergence.   First,  for  simplicity,  we  shall  consider 
Laplace's  equation  In  an  arbitrary,  bounded  lattice  region. 
We  shall  prove  that  It  Is  possible  to  select  the  acceleration 
parameter  so  that  a  suitable  norm  of  the  error  Is  reduced  after 
each  Iteration  by  the  amount   1  -  ch  +  0(h  ),   where  h   Is  the 
uniform  node  spacing  for  the  lattice,  and   c  >  0  depends  on 
only  the  minimal  eigenvalue  for  the  Laplace  operator.   We  shall 
Indicate  only  briefly  the  extension  of  these  results  to  elliptic 
equations  with  variable  coefficients. 
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Our  proof  of  the  convergence  of  the  alternating  direction 
methods  was  motivated  by  two  considerations.   First,  the  close 
resemblance  of  the  Iteration  equations  to  parabolic  difference 
equations^  and,  second,  an  Integral  estimate  technique  for 
proving  that,  under  suitable  conditions,  the  solutions  of 
parabolic  differential  equations  decay  exponentially. 

1.   Formulation  of  the  Problem. 

Let  oC    denote  the  (uniform)  lattice  of  side  h  determined 
by  the  nodes   (ah,ph),   a  and   p  being  Integers,  positive, 
negative,  or  zero.   Denote  by  ^  any  finite  subset  of  ^ .      In 
the  usual  way  (Bers  [5])>  we  decompose  S^  Into  two  disjoint 
subsets;   S,   the  Interior  nodes  of  "S,   and  9S,   the  boundary 
nodes  of  S. 

If   Q  is  any  subset  of  aC ,      we  denote  by  ^(Q)   the  linear 
space  of  all  real-valued  functions  on  oo  whose  support   Is 
contained  In  Q.   Note  that  ^(S)   Is  f Inlte-dlmenslonal  with 
dimension  equal  to  the  number  of  nodes  In   S.   The  advantage 
of  thus  trivially  extending  functions  on  Q  to  all  of  o'^    will 
become  apparent  later. 

We  denote  by  [\    the  usual  "five-point"  Laplace  difference 
operator.  I.e., 


^^  =  ^x3?+  ^yy  '  ^  '  ^(^)' 


In  general,  the  support  of  a  function  f  Is  the  closure  of 


the  set   {p|f(p)  ^   OJ. 
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where  the  subscripts  x     and  y  denote  forward  difference 
quotients,  and  3c  and  "y     denote  backward  difference  quotients 
(Cf.  Lees  [6J). 

We  shall  be  concerned,  first,  with  the  Dirichlet  problem 
for  the  elliptic  difference  equation 

(1.1)  Au  =  f   in  S, 

where  f  e  ^(S).   More  precisely,  we  seek  a  function  u  e  ^(S) 
that  satisfies  (l.l)  and  assumes  arbitrary  prescribed  values 
on  (^S. 

It  is  easy  to  see  that  this  problem  is  equivalent  to  the 
problem  of  solving  a  linear  system  of  algebraic  equations  of 
order  equal  to  the  nioraber  of  nodes  in  S.   Ey  means  of  the 
familiar  maximum  principle  for  the  difference  operator  /\  . 
one  can  show  that  this  system  of  linear  equations  always  has  a 
unique  solution.   In  this  paper  we  shall  be  concerned  with  the 
two  basic,  single-parameter  alternating  direction  methods  for 
the  inversion  of  this  system  of  equations. 

2.    The  Alternating  Direction  Methods. 

Let   w(0)  e  ^(S)      be  a  function  that  agrees  with  u  on 
3s,   and  otherwise  arbitrary  in  S;   w(o)   will  serve  as  the 
initial  approximation  to  u.   Now,  for  n  >^  1,   we  determine 
two  sequences  of  functions   [w(n)j   and  \y* in)j     in  (^("3) 
such  that 
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(2.1)  w(n)  =  w*(n)  =  u  on  ^S, 

and 


(2.2) 


(a)   p'^[w*(n)-w(n-.l)]  =  w*3^(n)  +  Wy-(n-l)  -  f 


(b)   p"^[w(n)-w*(n)]  =  ^^i^)   "  ^^-(n-l) 


In   S.   Here   p  >  0,   the  so-called  acceleration  parameter,  is 
to  be  selected  later.   These  equations  define  what  is  usually 
referred  to  as  the  Douglas-Rachford  alternating  direction  method 
for  solving  Z\u  =  f . 

In  [j)]    it  is  shown  that  (2.1)  and  (2.2)  determine  uniquely 
(alternately)  the  functions  w*(n)   and  w(n)   in  terms  of 
w(n"l)   and  the  values  of  u  on  5S.   This  operation  taking 
w(n~l)   into   w(n)   involves  only  the  inversion  of  approximately 
2  j/p   tridiagonal  matrices,  where  p   is  the  number  of  nodes  in   S. 

Following  Douglas  and  Rachford,  we  eliminate  from  (2.2)  the 
auxiliary  function  w*(n).   Adding  (2.2a)  to  (2.2b),  we  obtain 
the  equation 

(2.:5)  Dw(n)  =  w*3^(n)  +  Wy-(n)  -  f, 

where  we  have  Introduced  the  notation 


Dw(n)  =  p~  [w(n)-w(n-l)] 


Solving  (2.2b)  for  w*,   we  get 


w*(n)  =  w(n)  -  p  DWyy(n) 
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Hence, 


w*-(n)  =  w  -(n)  -  p^Al>w(n), 


XX  ^  '      XX ' 

where   A  is  defined  as  follows: 

/\u  =  u . 

f  ^  xxyy 

Putting  the  value  just  obtained  for  w*— (n)   into  (2.3),  we 
find  that  w  Is  a  solution  of  the  fifth  order  difference 
equation 

(2.4)  Dw  =  Aw  -  p^  /\Dw  -  f ,   in  S,   for  n  >  1. 

The  other  alternating  direction  method,  the  Peaceman- 
Rachford  method,  is  very  similar  to  this;  it  is  characterized 
by  the  equations 


p"'^[w*(n)-w(n-l)]  =  w*-(n)  +  Wy-(n-l)  -  f 


and 


p'"^[w(n)-w*(n)J  =  w*-(n)  +  Wy-(n)  -  f 


After  eliminating  from  these  equations  the  auxiliary  function 
w*(n),  we  find  that   w  is  a  solution  of  the  fifth  order 
difference  equation 

(2.5)  Dw  =  2Aw-pADw-   p^Adw  -   2f      in     S,      for     n  >  1, 

Our  problem,    now,    is   to   show  that     w(n)  — >•  u,      as     n  — >  go  , 
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;5.    Preparation. 

We  define  on  (^(S)   an  inner  product  <u,v>  as  follows; 


<u,v>  =  h^  \  u(P)v(P) 


P  e^ 

1/2 
The  norm  <v,v>     induced  by  this  inner  product  will  be 

denoted  by  ||v||  .   We  require  two  additional  norms  for  <*(S) 

11  V  111   =  (iv-lp  +  |lv-|p)^^2 
and 

l|v||2      =    (lIvlP   +   p2|^-_lp)^^2_ 


A  subscript  S  will  be  placed  on  these  quantities  whenever  we 
want  to  indicate  that  the  sum  is  to  be  extended  only  over  S. 

In  [7]  it  is  proved  that  the  minimal  eigenvalue   A  of  the 
Laplace  difference  operator  /S,      relative  to  S,      can  be  charac- 
terized as  follows: 

l|v||^    3 

A  =  inf  ^     . 

0  /  V   e  (S»(S)      ||v|r 

Since     HvlL    3  <.  ||vlL  ,      it  follows   immediately  that 


(3.1)  A||v||2   <    ||v|P, 


for  all     V   e  ^(S). 

On  the  strength  of  this  inequality,  we  now  prove  the 
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LEIVIMA  1.      Any  function     v  e  (^(S)      satisfies   the   Inequality 
0-2)  ilvll?   >   A Jlvll^ 


'1  ^  »o"^"2^ 


where 


(^•3)  K  = 


°        1   +   2Ap2h-2      • 
Proof.      We  have  from   (3-1)   and  the  definition  of     ||v|L     that 

llvlg   -   Allvll^-    Ap2llv__l|2. 

To  this  inequality  we  apply  the  elementary  inequality  [8] 

^^Kyil^   <  2l|v|li. 
valid  for  all     v  e   C(S),      and  we  find  that 

llvlg   -    A|lv|g2-   2ApV2||vl^, 

which  is  equivalent  to  {^.2), 

4.    Convergence  of  the  Alternating  Direction  Methods. 

Let  u  e  (i'CS)   be  the  solution  of  the  Dirichlet  problem 
for  Au  =  f  in  S,      and  let  w(n)   be  the  corresponding  solution 
of  the  Douglas-Rachf ord  equation  (2.4).   Then,  by  linearity,  the 
error  function  v(n)  =  u  -  w(n)   belongs  to  the  linear  space 
(^(S),   and  satisfies,  for  n  >^  1,   the  difference  equation 

(4.1)  Pv  =  Dv  -  Av  +  p^ADv  =  0 
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.   In  S.   Let  ^{n)   =  (1  +  pM-)"^,   where  m-  >  0.   One  verifies 
Immediately  that  ;2^  Is  a  solution  of  the  difference  equation 

(4.2)  Dfli  =  -  \ij6. 

THEOREM  1.   If  kL  <  Aq,   then  the  error  function  v(n)  for 
the  DouglaS"Rachf ord  method  satisfies  the  Inequalities 

l|v(n)l|^4/J(n)||v(0)||^    (1  =  1,2). 

Proof.   We  set 


n 


(4.3)  M  =  2p  y~^^(e)<v(0,Fv(0>. 

Since  v(^)  e  ^(S),   we  see  that  M  =  0. 

Introduce  the  function  z(4)  e  (^(S)   by  means  of  the  formula 

(4.4)  v(e)  =PU)zU)' 

Then  we  have  that 

Dv  =  0DZ  +  zT)^  -    plj^-  Dz 
which,  in  view  of  (4.2)  becomes 

Dv  =  ;Z^[aD2-M-z], 

where  a  =  jlT'^  (l)   >  0. 

From  this  and  (4.1)  it  follows  that 
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(4.5)  Pv  =  jZJ[aDz-ixz-Az+ap^  Adz-p^M,  A2]   =/S[aDz+Bz], 

where   B  Is  defined  in  the  obvious  way. 
Because  of  (4- 5), (4- 3)  becomes 

n 

(4.6)  M  ^  p  y~  2  <z(4),aDz(e)+Bz(0>. 

We  now  recall  from  [8]  the  following  quadratic  difference 
identities 


(4.7)  2<z,Dz>  =  D||zl|2   +   p||Dz||2 


(4.8)  2<z,Adz>  =  Dlz_-||2   +   p1|Dz_-||2, 


(4.9)  <z,Az>  =  -||z||2, 

and 

(4.10)  <Z,  AZ>  =ll2-yl|^. 

using  these  identities  in  (4.6),  we  obtain,  after  some 
slight  rearrangement,  the  identity 

(4.11)  M  =  p  ^  (aD||z||2+2(||z||2-^.|H|)   +  a2||Dz||2   +  ^.a||Dz||2 

i=l 

2         2 
Since  ||z||-|^  >  A  ||z||2,   by  lemma  1,   and  (x  •<  A  ,   we  have 

from  (4.11)  that 

M2  P^aD|lz||2   =  a||z(n)||2  -   allz(0)i2. 

1=1 
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But,  since  M  =  0,   we  obtain  from  this  and  (4.4)  that 

I|v(n)|i2  =0in)\\z{n)\\^^j6{n)\\zio)\\^   =  /^(n)I|  v(0)ll  g, 

which  proves  the  second  Inequality  of  the  theorem. 
Next,  we  set 

(^•12)  N  =  p^j2r2(Ol|Pv(Olls- 

FT 

Then,  by  (4.4)  and  (4.5),  (4.12)  becomes 

N  =  p  ^||aDz(0  +  Bz(^)ll^  . 
But 


aDz  +  Bz||2   =  a^llDzll  +  ||bz||2 


+  2a<Dz,Bz>2 

>  a^|Dz||^   +  2a<Dz,Bz>, 


where  we  have  dropped  the  subscript   S,   because  Dz  s  ^(S). 
Hence, 

(4.13)  .     N2  P  y~  I3.^\\^zf  +  2a<Dz,Bz>J. 

^=1  ^ 

Using  the  Identities  (4. 7)~(4 . 10)  again,  we  obtain  from 
(4.13)  the  Inequality 


N 


2  p  Yi  (^^11  ^^11^  +  ^^11^  I?  -  ^^^^ii^ii  +  ^f^i^^iii] 

e=i  ^ 
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Therefore,  by  adding  a  combination  of  the  above  inequalities, 


n      „ 
N  +  |xM  2  pa  \       Dlz|l^. 

Again,   N  +  p-M  =  0,   so  that 

l|2(n)llill|2(0)l|-L, 

which,  in  view  of  (4.4),  implies  that 

l|v(n)l|^4;^(n)l|v(0)ll-L. 

This  completes  the  proof  of  the  theorem. 

Since   p|j,  >  0,   theorem  1  implies  that  the  Douglas- 
Rachford  method  is  convergent,  but  we  also  have  the  stronger 
result: 

THEOREM  2 .   _I£  M-  =  A   and   p  =  /  \/Tk,      then  the  error 
function  v(n)   for  the  Douglas-Rachf ord  alternating  direction 
method  satisfies  the  inequalities 

(4.14)    llv(n)l|   <  (1  +N^  h)"''llv(0)||.        (i=l,2). 

Proof.   The  inequality  (4.l4)  follows  from  theorem  1,  by 
minimizing  }6{n)      as  a  function  of   p. 

In  an  analogous  fashion,  we  can  prove  the 
THEOREM  5.   If   IX  =  2Aq(1  -  >^qP)~^   an^  P  =  V\/2>r,   then 
the  error  function  v(n)   for  the  Peaceman-Rachf ord  alternating 
direction  method  satisfies  the  inequality 
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Ilv(n)||^  <  d   ••>^'^)"||v(0)l|^. 


Note  that  the  Peaceman-Rachford  method  has  a  smaller 
convergence  factor. 

The  preceding  results  can  be  extended  to  more  general 
elliptic  difference  equations,  e.g.,  to  those  that  come  from 
elliptic  differential  equations  of  the  form 

^(a(x,y)«iH)  +cL.(b(x,y)dii)  =  f(x,y) 
dx       <)x    c)y       ^y 

in  a  bounded,  open  set  Q,.      The  functions   a   and  b  must 
belong  to   c  (O.) .      The  main  point  that  we  wish  to  emphasize 
is  that,  because  the  inner  product  <u,v>  is  defined  as  a 
sura  over  all  of  oC ,      one  must  be  able  to  extend  a  and  b 
in  a  continuously  diff erentiable  way  to  a  slightly  larger  set 
CD  Z>  "fi.   This  can  be  carried  out,  for  instance,  by  means  of 
the  Whitney  extension  theorem  [9],    in  a  form  given  by 
H6rmander  [10]. 

One  can  also  show  that  similar  results  can  be  proved  for 
the  Douglas-Rachford  method  as  applied  to  elliptic  equations 
in  n   independent  variables,  although  this  is   not  the  case 
with  the  Peaceman-Rachford  method  [8]. 
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